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Abstract 


Researchers within the external Visibility System (XVS) element of the High-Speed 
Research (HSR) program have developed and evaluated display concepts that will provide the 
flight crew of the proposed High-Speed Civil Transport (HSCT) with integrated imagery and 
symbology to permit required path control and hazard avoidance functions while maintaining 
required situational awareness. The challenge of the XVS program is to develop and 
demonstrate operationally viable, economically feasible, and potentially certificated concepts 
that would permit a no-nose-droop configuration of an HSCT and expanded low visibility HSCT 
operational capabilities. The experiment described herein is one of a series of experiments 
exploring the “design space" restrictions for physical placement of an XVS display. In this 
study, the primary experimental issue examined was “conformality” of the forward display 
vertical position with respect to the side window in simulated flight conditions. “Conformality" 
refers to the condition such that the horizon and objects appear in the same relative positions 
when viewed through the forward windows or display and the side windows. 

In particular, this study quantified the effects of visual conformality on pilot flight path 
control and hazard avoidance performance. For this study, conformality related to the 
positioning and relationship of the artificial horizon line and associated symbology presented on 
the forward display and the horizon and associated ground, horizon, and sky textures as they 
would appear in the real view through a window presented in the side window display. The 
forward display symbology was presented as conformal (i.e., horizon at the same level as the real 
world view through the side window) or shifted up by 4 or 8 degrees visual angle. The potential 
incongruities in visual cues associated with nonconformality of the forward display had no 
significant performance consequences based on testing of six pilots in the NASA Langley Visual 
Motion Simulator (VMS). Scenarios evaluated included simulated approaches and landings, 
some with traffic and terrain hazards. No cases of simulator sickness or other physical signs of 
motion discomfort or vestibular effects were reported for any of the display conditions. Despite 
no significant negative performance consequences, when asked to rank the display conditions, 
the preference of five of the six pilots was for the conformal display condition. Factors to take 
into consideration in interpreting these findings include: ( 1 ) a limited vertical field of view 
(VFOV) of the XVS display due to hardware and experimental constraints, (2) a wide pillar 
separating the forward display and the side window (about 12 to 15 degrees), (3) the effect of 
limited vertical field of view on bank angle, (4) all hazards used in the experiment could be 
described as “slow onset,” (sudden, immediate, and large control inputs were not required for 
hazard avoidance), and (5) all hazard scenarios incorporated good visibility. Subjective ratings 
and rankings as well as numerous pilot comments are presented. 
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Introduction 


With air travel expected to double in the next five to 10 years, NASA and its U.S. aerospace 
industry partners are working to develop technologies for a future supersonic passenger jet 
referred to as the High-Speed Civil Transport (HSCT). As envisioned, this jet would carry 300 
passengers at more than twice the speed of sound, with ticket prices only 20 percent over today's 
comparable slower flights. Technology to make the HSCT possible is being developed as part of 
NASA's High-Speed Research (HSR) program. 

Researchers within the external Visibility System (XVS) element of the HSR program are 
developing and evaluating information display concepts that will provide the flight crew of the 
proposed HSCT with integrated imagery and symbology to permit required path control and 
hazard avoidance functions while maintaining required situational awareness. Researchers are 
tasked to develop and demonstrate operationally viable, economically feasible, and potentially 
certificated XVS concepts that would permit a no-nose-droop configuration of an HSCT and 
expanded low visibility HSCT operational capabilities. (See ref. 1.) The nose-droop mechanism 
currently used in the British-French Concorde provides the forward visibility required by the 
flight crew to adequately see the runway during landing and takeoff. The equipment needed to 
lower and raise the Concorde's nose adds weight and the nose in the lowered (or drooped) 
position adds drag. To be operationally viable, the HSCT design must be optimized to minimize 
weight and aerodynamic drag. The weight penalty of a nose-droop configuration for an aircraft 
the size of an HSCT is roughly estimated to be 10,000 pounds takeoff gross weight. (See ref. 2. ) 
Elimination of a hydraulic-powered mechanical nose on the HSCT by using an external visibility 
system (that would provide a capability equivalent to the forward facing windows in current 
commercial transport aircraft) would avoid the weight penalty associated with the nose-droop 
mechanism. The XVS concept does not have to provide a direct visual replacement for the 
forward windows, but it must enable the flight crew to perform the required functions of path 
guidance and hazard avoidance at the same levels provided by forward facing windows. (See ref. 
2.) 

The XVS will consist of a suite of sensors and supporting systems that will provide the flight 
crew information that would normally be available in a conventional cockpit through pilot vision 
in the forward direction. The current XVS concept consists of high-resolution video sensors, 
high-resolution XVS displays, navigation displays, weather radar with a traffic detection mode, 
the Traffic Alert and Collision Avoidance System, the Automatic Dependent Surveillance- 
Broadcast system. Automatic Surface Detection Equipment, and side windows with sunlight 
control systems. An initial assumption by the XVS element of the HSR program was that the 
XVS, in combination with any conventional side windows, would provide each pilot with a field 
of view (FOV) as least as great as the guidelines specified in ARP4101/2. (See ref. 3.) 
ARP4I01/2 is an SAE aerospace standards document detailing the requirements for pilot 
visibility from the flight deck. To satisfy the criteria of the ARP4 101/2 vision envelope, the 
current display configuration consists of one XVS display each for the pilot and co-pilot, each 
containing a 40° horizontal and 50° vertical FOV. The XVS display uses 60 pixels per degree 
(assumed to approach the practical limit of human eye perception). The forward visibility 
provided by the XVS display is augmented by natural vision through the side windows. (See ref. 
4.) 
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Flight and ground vehicle tests of XVS technologies have been conducted as part of the HSR 
Flight Deck research project. In 1995, pilots flew approximately 90 approaches and landings 
from the NASA 737's windowless research cockpit. The pilots were required to control and land 
the aircraft relying only on sensors and computer-generated images (including various symbols) 
on the XVS display. This first XVS flight test gave researchers confidence that a future 
supersonic passenger jet could indeed be flown without forward facing windows in the cockpit. 

High priority research issues, like display size and conformality (ref. 5), were identified by 
researchers within the XVS element. Results from a 1996 XVS flight test (ref. 6) helped define 
the XVS display size requirements (40° horizontal by 50° vertical FOV). This flight test 
provided data on the effect of decreasing the inboard FOV by 10° (from 50° to 40°) on pilot path 
control. Based upon the structural constraints on the HSR Technology Concept Airplane (TCA) 
flight deck, it may be required that the XVS display be vertically positioned or rotated (in the 
pitch axis) to fit the space available. This creates a situation such that the forward and side views 
may be “nonconformal.” That is, the horizon and symbology on the XVS display in front may 
appear to be displaced relative to the real world view from the side window. This conformality 
displacement may adversely affect pilot flight path and collision avoidance performance. 

The experiment described herein is one of a series of experiments exploring the “design 
space” restrictions for placement of an XVS display. “Design space” refers to the space 
available for placement of an XVS in the flight deck of an HSCT while accounting for structural 
constraints. In this study, the primary experimental issue examined was conformality of the 
forward XVS display vertical position with respect to the side window in simulated flight 
conditions. In a conformal display, distant display images are displayed at the correct size and 
location angularly as their real world sources would be if viewed through a window. In 
particular, this study quantified the effects of visual conformality on pilot flight path control and 
hazard avoidance performance. For this study, conformality related to the positioning and 
relationship of the artificial horizon line and associated symbology presented on the forward 
XVS display and the horizon and associated ground, horizon, and sky textures as they would 
appear in the real view through a window presented in the side window display. The side 
window, though computer generated, represented a real window and a view of the real world. 
For this experiment, the forward XVS display symbology was presented as conformal (i.e., 
identical to the real world view through the side window) or shifted up by 4 or 8 degrees visual 
angle. 

Due to hardware limitations in the simulator, the forward XVS display used in this experiment 
had only a 22° vertical FOV at the pilot eye reference point instead of the required 50° vertical 
FOV as determined by ARP 4101/2. It was not possible to achieve “true” nonconformality by 
physically repositioning the forward display monitor because of constraints with regard to the 
physical structure of the simulator. Therefore, nonconformality was created artificially by 
vertically positioning the “appearance” of the display on the forward monitor. To maintain a 
consistent vertical FOV across conformal and nonconformal displays, a 14° vertical FOV was 
chosen. Hence, an 8° shift in the visual angle (22° vertical space available minus fixed 14° 
vertical FOV ) was the maximum amount of nonconformality that could be studied in this 
experiment. 

In the present study, symbology overlaid the forward scene information analogous to 





symbology found on Head-up displays (HUDs). HUDs spatially overlay instrument symbols 
(e.g., a horizon line) conformally with the outside world. In reference 7, Fadden et al. state that 
several flight simulation studies have demonstrated that conformal HUDs are beneficial when 
compared to head-down displays (HDDs). It is hypothesized that a conformal XVS display is a 
more natural way for a pilot to view an external scene as it is comparable to flying heads-up, 
eyes-out the window. In reference 8, the authors describe some potentially significant human 
factors problems related to the optical geometric conformality of a display. They state that, 
“perfect geometric conformality is achieved when the locations of the imaged objects within the 
virtual space register exactly with the optical locations of the real objects as directly viewed by 
the observer.” The authors describe seven critical geometric display conformality disruptions. 

The one appropriate for discussion within the context of the current experiment is classified as 
“display displacement.” This geometric conformality disruption exists when the display is not 
directly in front of the pilot so that the horizon line is not at eye level. The authors warn that the 
misplacement of the horizon line combined with vestibular cues given by the center of gravity 
could generate a cue conflict that may lead to physiological side effects particularly for 
significant longitudinal accelerations or decelerations as they are known to create pitch attitude 
illusions. If this cue conflict exists, a pilot could have trouble distinguishing between the illusion 
of pitch attitude due to a misplaced horizon or due to acceleration effects. The authors state that, 
assuming no large visual/vestibular effects exist, a pilot can often learn to use a nonconformal 
display effectively. 

The primary objective of this simulator experiment was to quantify the effects of visual 
conformality on pilot flight path control and hazard avoidance performance. Specifically, the 
hypotheses tested were: ( 1 ) Pilot flight control performance and avoidance of airborne hazards 
will degrade with nonconformal displays as compared to conformal displays due to incongruities 
in visual and motion cues associated with nonconformality; (2) No differences in pilot flight 
control performance will be found between motion and non-motion simulator trials, meaning that 
non-motion simulation facilities could be used in further conformality experiments; and (3) 

Pilots will prefer conformal XVS displays to nonconformal XVS displays. 

To address the second hypothesis concerning the effect of motion cues, all path control trials 
were conducted under both motion and no-motion conditions. If pilot performance was found to 
not differ across these motion conditions, then there is no requirement to include motion in future 
XVS studies, saving time and money and significantly increasing the number of facilities 
available to support future experiments. 


3 



Method 


Subjects 

Subjects were six transport aircraft-rated pilots (one Captain, five First Officers) recruited 
under contract by Lockheed-Martin from three airlines: United, American, and USAirways. All 
subjects were paid for their participation and transportation and lodging expenses were also paid. 
The number of years flying commercial aircraft that subjects reported ranged from six to 1 8, with 
a mean of 9.75 years. Three of the subjects also had experience flying military aircraft (for 10, 

20 and 21 years, for the three subjects respectively). The total number of hours flying ranged 
from a minimum of 3,700 to a maximum of 1 7,000+, with a mean of 8,540 hours flying. The 
total number of hours flying as pilot in command ranged from 3,200 to a maximum of 1 5,000+ 
hours, with a mean of 5,300 hours. Two of the pilot subjects reported that they had no 
experience in “glass cockpit” aircraft; three reported one to five years' experience, and one pilot 
reported six+ years of experience with glass cockpits. The pilot subjects' flight experience and 
types of aircraft flown are summarized in Appendix A. 

Simulator 

This experiment was conducted in the NASA Langley Research Center's Visual Motion 
Simulator (VMS), a generic flight simulator that can be custom-configured for a variety of 
aircraft-related experiments. The VMS rests on a six degree-of-ffeedom motion platform and 
has a flight deck with generic controls and displays. The simulator is outfitted to support 
research in both transport aircraft and helicopters. The simulation software driving the simulator 
for this experiment was the HSR Reference H, cycle 2B. This simulation package was chosen as 
it was already implemented in the VMS and also incorporated a baseline HUD that only required 
slight modifications to the symbology in order to serve as the XVS display (primary flight 
display) described below. 

For this experiment, pilot subjects sat in the left seat and the experimenter sat in the right seat 
during experimental trials. The repositionable left seat is equipped with a sidestick controller 
(for pitch and roll control) and conventional rudder pedals (for yaw control) with toe brakes. 

The sidestick controller rests on the left armrest of the left seat. A four-lever throttle quadrant 
(with a back-driven autothrottle), a Control and Display Unit (CDU), and gear/flap levers are 
located on the console between the left and right seats. During experiment trials, the autothrottle 
was in operation and only the left-most two of the four throttle levers were used. A photograph 
of the VMS cockpit interior (pilot side) is shown in Figure 1. 

The VMS has two forward windows and two side windows, each with a wide-angle 
collimated display providing out-the-window views. Collimation is achieved via commercially 
available systems that utilize a mirror-beam-splitter arrangement (See Figure 2). The total FOV 
of the forward display from the left seat is 40 degrees vertically and 50 degrees horizontally. 

The out-the-window scenes are computer-generated images (CGI) created by an Evans & 
Sutherland graphics engine. The images are constructed from database information; for this 
experiment, the out-the-window views were of the Denver International Airport area. 
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The XVS concept was used as the primary flight display in this experiment and contained 
guidance elements and basic flight condition information. The XVS symbology was overlaid 
onto the computer generated out-the-window scene which, in turn, was displayed to the pilot on 
the collimated forward window display. The specific XVS symbology chosen for display in this 
experiment is shown in Figure 3. A detailed description of the original HSR Reference H model 
and associated baseline HUD symbology can be found in Reference 9. A sidestick XVS 
symbology decluttering button permitted removing some, all, or none of the overlaid XVS 
symbology. 

The HDD's presented airspeed, altitude, and vertical speed on traditional round dial 
instruments. A CRT display presented Attitude/Pitch information, while engine performance 
was shown on round dial indicators. 

The VMS XVS software was modified to accommodate this experiment. The forward images 
were electronically masked and also shifted upward to produce the conformality/ non- 
conformality display conditions which are described in detail in the next section. 



Figure 1. Visual motion simulator cockpit interior. 
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Figure 2. Optical eollimation system based upon a mirror-beam-splitter arrangement. 
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Display Conformality/Nonconformality 

When the world is viewed during flight through an aircraft flight deck's front and side 
windows, the views are the same. That is, the horizon appears in the same position in both 
views; when this occurs, the view may be said to be “conformal.” With the XVS system, the real 
world information that is displayed is provided through the integration of synthetic, sensor, and 
camera-based inputs. Based upon the structural constraints on the HSR Technology Concept 
Airplane (TCA) flight deck, it may be required that the XVS display be vertically positioned or 
rotated (in the pitch axis) to fit the space available. This creates a situation such that the forward 
and side views may be “nonconformal.” That is, the horizon and symbology on the XVS display 
in front may appear to be displaced relative to the real world viewed from the side window. This 
“conformality displacement” may adversely affect pilot flight path and collision avoidance 
performance. There is no guidance from the research literature that relates to possible effects of 
display conformality/nonconformality on pilot performance. Therefore, the present study was 
designed to address this issue and to provide a “reference point” with regard to pilot performance 
under conformality and varying degrees of nonconformality. In particular, these studies explored 
restrictions on the design space of the XVS by evaluating vertical displacements of the XVS 
display and assessing the impact of the created nonconformality on pilot performance. 
Conformality/nonconformality created by vertical displacement of the display monitor is 
depicted in Figure 4. 


Side Window 




Conformal 






— 




8° Hiaher 


4° Higher 
Nonconformal 


Nonconformal 


Side Window 


Figure 4. Conformality created through vertical displacement of the display monitor. 

There were four conformality/nonconformality display conditions in this study. These 
conditions are summarized in Table 1 and graphically depicted in Figure 5. In the present 
experiment, it was not possible to achieve “true” nonconformality by physically repositioning the 
forward display monitor because of constraints with regard to the physical structure of the 
simulator. Therefore, nonconformality was created artificially by vertically positioning the 
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“appearance” of the display on the forward monitor. Two vertical shifts to simulate 
nonconformality were chosen for exploration in the present study: four degrees and eight 
degrees. 

In addition, the total vertical display area of the forward monitor was 40°, but at the pilot 
design reference point, the vertical field of view (VFOV) subtended an angle of 22°, with 7° 
devoted to the image of the ground at the base of the display area. For consistency, a ground 
display area of 7° was maintained across all displays. To achieve this, an opaque black mask was 
placed across the bottom of both the 4° and 8° nonconformal displays. 

As the image was shifted vertically (by 4° or 8°) to achieve nonconformality and the mask was 
placed at the bottom of the display, the total VFOV became shortened (for a VFOV of 1 8° and 
14° for the 4° and 8° nonconformal shifts, respectively). This confounded the effects of 
nonconformality and VFOV. That is, the non-conformal displays were both nonconformal and 
had a shortened VFOV relative to the conformal display. To maintain a consistent VFOV across 
conformal and nonconformal displays, a 14° VFOV was maintained by placing a black mask 
across the top of the display (if required to maintain a 14° VFOV). This created three display 
types, all with a 7° ground area and a 14° VFOV: ( 1 ) conformal, (2) 4° nonconformal, and (3)8° 
nonconformal. 

The masks significantly reduced the possible VFOV. Pilot performance may be affected by 
nonconformality or by a significantly reduced VFOV. To examine the effects of 
nonconformality independent of the reduced VFOV, a fourth display condition was created such 
that the display was at maximum non-conformality (i.e., 8°), but the display was not masked, 
allowing the pilots a full view of the display area. In this display condition, the total VFOV was 
22°, the maximum VFOV allowable. 

Table 1. Conformality/Nonconformality Display Conditions 


Conformality Condition 

Vertical Displacement 
(degrees) 

Vertical Field of View 
(degrees) 

(1) Conformal 

0 

14 

(2) Nonconformal 4 deg 

4 

14 

(3) Nonconformal 8 deg 

8 

14 

(4) Nonconformal 8 deg+ 

8 

22 


Scenarios 

In the present study, multiple scenarios were created to assess the effects of conformality/ 
nonconformality on pilot obstacle and terrain collision avoidance performance. In some 
scenarios, the pilot is required to control the aircraft while following another aircraft; in other 
scenarios, an obstacle (another aircraft) is unexpected; and in another scenario, the task is to 
avoid collision with terrain. These scenarios are summarized in Table 2 at the end of this 
section. There were four types of scenarios: ( 1 ) Flight Path Control, (2) Hazard Avoidance, (3) 
Terrain Avoidance, and (4) Training 
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Flight Path Control and Hazard Avoidance Scenarios 

To assess pilot flight path control performance, two “base” scenarios were created that 
required the pilots to perform multiple flight path control maneuvers (the differences in the two 
scenarios did not allow direct comparison of pilot performance across the scenarios). Both 
“base” scenarios required the pilot to perform turns during which the horizon appeared to 
traverse (from the pilot's viewpoint) from one window to the other (that is, front window to side 
window or side to front). 

To assess hazard avoidance , variants were created from each of the two “base” scenarios. 
There were two scenario sets, each set composed of a “base” scenario (for flight path control 
assessment) and its variants (for hazard avoidance assessment). 
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Figure 5. Conformality/nonconformality displays with vertical fields of view. 
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Scenario Set A: 


• Scenario 1 : The first “base” scenario (Scenario I ), created to asses s flight path control 
performance, required pilot subjects to follow a lead aircraft to a landing and make 
several left turns while following the lead aircraft, continuing through touchdown, 
landing on Rwy 35L, and runway turnoff. On initiating the scenario, the pilot's aircraft 
was positioned 7.6 nautical miles from the runway, at an altitude of 2000 feet and with an 
airspeed of 155 knots. (See Figure 6 for a graphical depiction of Scenario 1 .) 


Variants of “base” Scenario I to assess Ha^i^_Ayoidance; . 

• Scenario 2 : In Scenario 1 , the pilot was required to follow the lead aircraft, making 
several left turns, then land on Rwy 35L. In Scenario 2, the initial conditions were 
identical to those in Scenario 1 and the pilot was required to perform the same maneuver, 
bringing the aircraft in-trail behind the lead aircraft. However, on first turning to come 
in-trail behind the lead aircraft, a second aircraft 300 feet below Ownship in-trail behind 
the lead aircraft is encountered. The pilot is required to perform an avoidance maneuver 
to avoid the second in-trail aircraft (see Figure 7). 

• Scenario 3 : In Scenario 1 , the pilot was required to follow the lead aircraft, making 
several left turns, then landing on Rwy 35L. In Scenario 3, the initial conditions were 
identical to those in Scenario 1 and the pilot was required to perform the same maneuver, 
bringing Ownship in-trail behind the lead aircraft. However, on first turning to come in- 
trail behind the lead aircraft, the lead aircraft slows speed. The pilot is required to 
perform an avoidance maneuver to avoid the lead aircraft (see Figure 8). 
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Scenario 1 (Flight Path Control) 

Downwind to 35L, follow B-747, Land 

(Not to Scale) 



Figure 6. Scenario 1 (Flight Path Control evaluation). 


Scenario 2 (Hazard Avoidance: Other Traffic) 

Downwind to Rwy 35L, follow B-747, Avoid Traffic 

(Not to Scale) 



Figure 7. Scenario 2 (Hazard Avoidance evaluation): a variant of Scenario 1 . 







Scenario 3 (Hazard Avoidance: Other Traffic ) 

Downwind to Rwy 35L, follow B-747, Target Slows 

(Not to Scale) 




Ownship 



Initial conditions: 
DME 7.6 nm 
alt 2000 ft 
airspeed 155 kts 


Lead Aircraft 


i 

i 

i 

|35R| 





Figure 8. Scenario 3 (Hazard Avoidance evaluation): a variant of Scenario 1 . 

Scenario Set B : 

• Scenario 4 : The second “base” scenario (Scenario 4) to assess flight path control 
performance required pilot subjects to perform a landing with a four-mile straight-in final 
approach to Rwy 35L, with a small transport aircraft on the left, and a large transport 
aircraft on the right landing on Rwy 35R. The scenario continued through landing on 
Rwy 35L and runway turnoff. The pilot’s aircraft (“Ownship”) was initially at a distance 
of 4.7 nautical miles from the runway and at an altitude of 1500 feet, with an airspeed of 
155 knots. The aircraft on the left (small transport) was initially a distance of 4 nautical 
miles from the runway, climbing at 2,000 feet, with an airspeed of 155 knots. (See 
Figure 9 for a graphical depiction of Scenario 4.) 

Variant of “base” Scenario 4 to assess Hc^jd^Avojdgnce : 

• Scenario 5: In Scenario 4, the pilot was required to perform a landing with a four-mile 
straight-in final approach to Rwy 35L, with two parallel transport aircraft (one on the 
right, landing on Rwy 35R, and one on the left). To assess hazard avoidance , in Scenario 
5, the transport aircraft on the right performs an excursion to the left, crosses the 
Ownship flightpath and heads to land on Rwy 35L. Go-around instructions are 
immediately given to the pilot subject by ATC (to climb to an altitude of 1500 feet and 
turn to a heading of 330) and the pilot subject is required to perform the maneuver to 
avoid the right aircraft (see Figure 10). 
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Scenario 4 (Flight Path Control) 

Final to 35L, Traffic Right, Traffic Left 

(Not to Scale) 


Initial Conditions: 
DME 4.7 nm 
alt 1500 ft 
airspeed 155 kts 


i 

i 

i 

35L 

llll 


I 

I 

I 

35R 

llll 


Aircraft #2 
DME 4.0 nm 
alt 2000 ft, climbing 
airspeed 155 kts 



Aircraft #1 
Landing 35R 


Ownship 


Figure 9. Scenario 4 (flight path control evaluation). 


Scenario 5 (Hazard Avoidance: Other Traffic) 

Final to Rwy 35L, Traffic Right Blunders, Traffic Left 


(Not to Scale) 



Figure 10. Scenario 5 ( Hazard Avoidance evaluation): a variant of Scenario 2. 

Therefore, five scenarios were created to assess pilot flight path control and hazard 
avoidance performance, organized into two sets: Scenario Set A consisted of a base scenario 
(Scenario 1 ) and two variants of this base (Scenarios 2 and 3), and Scenario Set B consisted of a 
base scenario (Scenario 4) and a variant of this base (Scenario 5). 
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Terrain Avoidance Scenario 


In addition, a sixth scenario was created to assess pilot terrain avoidance performance 
under conformality and nonconformality display conditions. This scenario (Scenario 6) was 
initiated with the pilot subject's aircraft at an altitude of 6,000 feet, an airspeed of 155 knots, and 
a heading of 340°, flying above a mountainous area with mountain ridges both to the left and 
right. The pilot was instructed to turn to a heading of 270°, toward the left mountain ridge, and 
fly above the ridge, maintaining an altitude of 500 feet above the ridge (see Figure 1 1 ). 


Scenario 6 (Hazard Avoidance: Terrain) 




Figure 1 1. Scenario 6 (Terrain Avoidance evaluation). 
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Training Scenarios 

Prior to having the pilot subjects perform the Flight Path Control, Hazard Avoidance, and 
Terrain Avoidance scenarios, they were trained on several aspects of the aircraft simulation, the 
experiment procedures, and the data collection protocol. The training trials also provided the 
pilot subjects landing and flare practice (see the Procedures section for a detailed description of 
the training process). Two scenarios were developed specifically for this training. 

The first training scenario (Training Scenario 1 ) began at 500 feet altitude and the pilot 
subjects were required to perform a straight-in final approach and landing on Rwy 35L. The 
second training scenario (Training Scenario 2) was initiated at 1500 feet altitude and 
approximately five miles out on the base leg, and the pilot subjects were required to turn onto 
final and complete the landing on Rwy 35L. Training Scenarios 1 and 2 are given in Figures 12 
and 13. 


Training Scenario 1 

Straight-in Final to Rwy 35L, Land 

(Not to Scale) 




Initial Conditions: 
alt 500 ft 
airspeed 155 kts 



Ownship 


Figure 12. Training Scenario 1. 
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Training Scenario 2 


Base, Turn to Final to Rwy 35L, Land 

(Not to Scale) 


Ownship 


Initial conditions: 
DME 5 nm 
alt 1500 ft 
airspeed 155 kts 
Base leg 


I 

l 

I 


A 


8 

c*> 



Figure 13. Training Scenario 2. 


Table 2. Summary of Flight Path Control, Hazard Avoidance, Terrain Avoidance, and Training 

Scenarios 


Scenario Number and 
Type 

Scenario Description 

Performance 

Evaluated 

Scenario Set A: 



( 1 ) Base Scenario 

Follow lead a/c through turns to landing 

Flight Path Control 

(2) Variant of Scenario 1 

Follow lead a/c through turns to landing; 
avoid aircraft in-trail behind lead aircraft 

Collision Avoidance 
(other traffic) 

(3) Variant of Scenario 1 

Follow lead a/c through turns to landing; 
when in-trail, avoid lead aircraft which 
slows airspeed 

Collision Avoidance 
(other traffic) 

Scenario Set B: 



(4) Base Scenario 

Four-mile straight-in final to Rwy 35L; 
small transport on left; large transport on 
right landing on Rwy 35R 

Flight Path Control 

(5) Variant of Scenario 4 

Four-mile straight-in final to Rwy 35L; 
small transport on left; avoid large 
transport on right making excursion 
across flight path to land on Rwy 35L 

Collision Avoidance 
(other traffic) 




Terrain Avoidance 



(6) Base Scenario 

Fly through valley between mountains; 
turn left, fly over top of ridge, climb to 
clear ridge top, level-off, maintain 500 
foot altitude over ridge 

Collision Avoidance 
(terrain) 

Training 



Training Scenario 1 

Straight-in Final to Rwy 35L, land 

None 

Training Scenario 2 

Base, turn to Final to Rwy 35L, land 

None 
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Experiment Design 
Independent Variables 


Flight Path Control and Hazard Avoidance Evaluation. The flight path control and hazard 
avoidance evaluation was a 4 x 2 x 6 (Conformality x Motion x Subjects) factorial design using 
two sets of scenarios (A and B). Each trial involved manipulating the type of display 
conformality while performance data were collected as pilot subjects flew pre-defined scenarios. 
Each trial consisted of a unique combination of conformality, motion, and scenario. The 
evaluation design is summarized in Table 3. 

For each of the two sets of scenarios (A and B), the base scenarios were conducted under both 
motion and no motion conditions, while the variant scenarios were conducted with motion only. 
That is, for Scenario Set A, pilot subjects performed Scenario 1 under motion and no motion 
conditions, while Scenarios 2 and 3 were conducted only with motion. For Scenario Set B, 
Scenario 4 was conducted under motion and no motion conditions, while Scenario 5 was 
conducted under motion only. 

In addition, trials were replicated. The base scenario trials (Scenarios 1 and 4) received three 
replications (to allow direct comparisons of pilot performance across both Motion and No 
Motion conditions of these specific flight control evaluations) and the remaining scenario trials 
(Scenarios 2, 3, and 5) received two replications each. All six pilot subjects performed all trials 
across all scenarios, conformality types, and motion conditions. Therefore, there were a total of 
24 trials for each base scenario (4 conformality types x 2 motion conditions x 3 replications) and 
a total of nine trials for each of the three variant scenarios (4 conformality displays x 1 motion 
condition x 2 replications). 

Table 3. Experiment Design: Flight Path Control and Hazard Avoidance Performance 

Evaluation 


Factor 

Factor 

Levels 

Description of Factor Levels 

( 1 ) Conformality Type 

4 

Conformal 22 Degrees VFOV 
4 deg Non-conformal 14 deg VFOV 
8 deg Non-conformal 14 deg VFOV 
8 deg Non-conformal 22 deg VFOV 

(2) Motion/No Motion 

2 

Motion/No Motion (Scenarios 1 & 4) 


1 

Motion only (Scenarios 2, 3, & 5) 

(3) Subjects 

6 

Six transport pilots 

(4) Replications 

3 

Base scenario (Scenario 1 ) 


2 

Hazard Avoidance scenarios (Scens 2, 3, & 5) 


Terrain Avoidance Evaluation. The terrain avoidance performance assessment was 
carried out only under motion conditions, with one trial at each of the four conformality/ 
nonconformality display conditions with no replications. All terrain avoidance trials were 
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performed by all of the six pilot subjects. Therefore, the terrain avoidance evaluation was a 4 x 6 
factorial design (Conformality x Subjects) having a total of four trials for each subject (4 
conformality displays x motion x no replications). This design is summarized in Table 4. 

Table 4. Evaluation Design: Terrain Avoidance Evaluation (Scenario 6) 


Factor 

Levels of 
Factor 

Description of Factor Levels 




( 1 ) Conformality Type 

4 

Conformal 22 Degrees VFOV 
4 deg Non-conformal 14 deg VFOV 
8 deg Non-conformal 14 deg VFOV 
8 deg Non-conformal 22 deg VFOV 

(2) Subjects 

6 

Six transport pilots 


Dependent Measures 

A detailed list of all dependent measures is given in Appendix B. Several types of objective 
data were automatically collected on all trials to indicate experimental evaluation condition (e.g., 
scenario number, conformality condition), the aircraft state, and the pilot subject’s performance. 
Aircraft state data were collected at variable rates; data were recorded at 1 Hz [cycles per 
second] from the start to the end of the simulator run and at 20 Hz from five nm to touchdown. 

Aircraft state data collected included: aircraft position; aircraft attitude; indicated airspeed; 
distance to other aircraft; lateral and vertical path error; pitch, roll, and sink rates; controller 
input angles; rudder pedal position; throttle position; and engine power. 

In addition, some aircraft state measures were taken as single event measures at touchdown. 
At first gear touchdown, aircraft position, elapsed time, sink rate, lateral velocity, and indicated 
airspeed were recorded. At nosewheel touchdown, aircraft position, elapsed time, lateral 
velocity, and pitch rate were recorded. 

In addition to aircraft state measures, both heart rate and electro-oculogram (EOG) data were 
recorded for the pilot subjects. Heart rate was recorded by an unobtrusive earclip plethysmo- 
graph and EOG was recorded by a unobtrusive sensors mounted on skin adjacent to the eye to 
detect “left looks.” 

Several subjective measures were also taken at selected points throughout the experimental 
trials. These measures were in the form of the NASA TLX instrument for estimating perceived 
workload, preference rankings of the conformal/nonconformal displays taken at one time at the 
end of all experimental data collection runs, and a questionnaire constructed to subjectively 
compare display “usability" across conformality conditions. All test sessions were also 
videotaped. 
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Trials 


All Training, Flight Path Control, Hazard Avoidance, and Terrain Collision Avoidance trials 
are summarized in Table 5. 

Training . During training trials, the pilot subjects were required to fly the two training 
scenarios, giving them basic practice in handling and landing the aircraft. Subjects flew four 
initial training trials with the conformal display (two trials with each of the two training 
scenarios), then four trials at each of the four conformality conditions (two with each training 
scenario). All training trials were conducted with no motion. This totaled 20 training trials, ten 
with each of the two training scenarios (eight with the conformal display [0°] and four each with 
the non-conformal displays [4°, 8°, 8°+] ). 

Flight Path Control and Hazard Avoidance . The flight path control/hazard avoidance 
evaluations required the pilot subjects to fly the scenarios at all four conformality display 
conditions. Within each evaluation, a single simulator trial involved a unique combination of 
scenario, motion, and conformality; a trial was replicated either two or three times, depending on 
the scenario type. 

In evaluating performance with Scenario Set A, the pilot flew Scenarios 1, 2 and 3. Pilot 
subjects flew the base scenario (Scenario 1 ), both with motion and no motion with three 
replications, and the hazard avoidance scenarios (Scenarios 2 and 3) with motion only and with 
two replications (to reduce the total number of trials). 

In evaluating performance with Scenario Set B, the pilot subjects flew Scenarios 4 and 5. 
Subjects flew the base scenario (Scenario 4), both with motion and no motion with three 
replications, and the collision avoidance scenario (Scenario 5) with motion only and with two 
replications (again, to reduce the total number of trials). 

Also, it was particularly important to assess the impact of conformality display type on 
performance on the first trial involving an “unexpected” hazard avoidance situation (i.e., with 
Scenarios 2, 3, and 5). Therefore, a single extra trial of each “collision avoidance” type was 
embedded in the set of trials where they were first encountered for each subject. Therefore, there 
is one extra trial of each of these three hazard avoidance scenario conditions. 

Therefore, for Scenario 1, this resulted in a total of 12 Motion (4 conformality x 3 
replications) trials and 12 No Motion (4 conformality x 3 replications) trials. There were nine 
trials for Scenario 2 (4 conformality x 2 replications + one extra trial for the first hazard 
avoidance trial) and nine trials for Scenario 3 (4 conformality x 2 replications + one extra trial 
for the first hazard avoidance trial). Therefore, this provides a total of 24 Scenario 1 trials, and 
nine trials each for Scenarios 2 and 3, totaling 42 trials when evaluating pilot subject 
performance with Scenario Set A. 

Following the same pattern, the performance evaluation with Simulation Set B resulted in a 
total of 12 Motion (4 conformality x 3 replications) trials and 12 No Motion (4 conformality x 3 
replications) trials for Scenario 4. Scenario 5 had a total of nine trials (4 conformality x 2 
replications + one extra trial for the first hazard avoidance trial). Therefore, this provided a total 
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of 24 Scenario 4 trials and nine Scenario 5 trials, totaling 33 when evaluating pilot subject 
performance with Scenario Set B. 


Terrain Avoidance . This evaluation required the pilot subjects to fly Scenario 6 only. This 
was carried out under Motion only, with one trial at each of the four Conformality conditions, 
creating a total of four Terrain Avoidance trials. 

Table 5. Summary of Trials by Evaluation Condition, Scenario Set, Motion, Conformality, and 

Replications. 


Experiment and Scenario 

Motion 

Conformality 

Replications 

Number 



Condition 


of Trials 

Training 





Initial Training: 
Training Scenario 1 

No Motion 

0° 

2 

2 

Training Scenario 2 

No Motion 

0° 

2 

2 

Training Scenario 1 

No Motion 

0°, 4°, 8°, 8°+ 

2 

8 

Training Scenario 2 

No Motion 

0°, 4°, 8°, 8°+ 

2 

8 

TOTAL Number of Trials 
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Flight Path Control & Hazard 
Avoidance (Other Traffic) 
Evaluation 





Scenario Set A 





Base scenario (Scenario 1 ) 

Motion 


3 




0°, 4°, 8°, 8°+ 

3 

Bi 

Collision Avoidance (Scenario 2) 

Motion 

0°, 4°, 8°, 8°+ 

2 

8(+l) 

Collision Avoidance (Scenario 3) 

Motion 

0°, 4°, 8°, 8°+ 



TOTAL Number of Trials 




42 






Scenario Set B 





Base scenario (Scenario 4) 

Motion 

0°, 4°, 8°, 8°+ 

3 



No Motion 

0°, 4°, 8°, 8°+ 



Collision Avoidance (Scenario 5) 

Motion 

0°, 4°, 8°, 8°+ 

2 

BSH 

TOTAL Number of Trials 




33 






Terrain Collision Avoidance 





Scenario 6 

Motion 

0°, 4°, 8°, 8°+ 

None 

4 

TOTAL Number of Trials 




4 






TOTAL Experimental Trials 




79 
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Organization of Trials 

A typical order of all trials for one pilot subject is given in Appendix C. 

Training Session. Pilot subjects received a total of 20 training trials, ten with each of the two 
training scenarios. The first four trials were conducted using the conformal display condition 
(for initial familiarization). The remaining 16 trials were blocked by conformality and given in 
the order ( 1 ) conformal, (2) 4 degree nonconformal, (3) 8 degree nonconformal, and (4) 8+ 
degree nonconformal, with four trials at each conformality condition (two with each of the two 
training scenarios). 

Flight Path Control and Hazard Avoidance (Other Traffic) Evaluations. All Flight Path 
Control and Hazard Avoidance (Other Traffic) Evaluations using Scenario Sets A and B were 
conducted separately. Within both sets of evaluations, trials were blocked by Motion (with three 
of the subjects receiving all Motion trials first, followed by No Motion trials, and the other three 
subjects receiving the opposite order). Within both evaluations, trials were also blocked by 
conformality (0°, 4°, 8°, 8°+) and the orders of conformality trial blocks were counterbalanced 
across subjects (ordering was determined using a balanced 4x4 Latin Square). 

For evaluations with Scenario Set A, Scenarios 1 , 2, and 3 trials were randomized within a 
conformality block. Therefore, with Scenario Set A, pilot subjects received a total of 1 2 
Scenario 1 trials under the No Motion condition, blocked by and counterbalanced across 
conformality. Under Motion, subjects received 1 2 Scenario 1 trials, 9 Scenario 2 trials, and 9 
Scenario 3 trials, blocked by and counterbalanced across conformality. 

For evaluations conducted with Scenario Set B, Scenarios 4 and 5 trials were randomized 
within a conformality block. Subjects received a total of 12 trials with Scenario 4 under the No 
Motion condition, blocked by and counterbalanced across conformality. Under the Motion 
condition, they received 1 2 Scenario 4 trials and 9 Scenario 5 trials, blocked by and 
counterbalanced across conformality. 

Terrain Avoidance Evaluation. Subjects received the four Scenario 6 trials in one block, with 
the ordering of the four conformality conditions counterbalanced across subjects (with ordering 
determined by a 4 x 4 Latin Square). 

Procedure 

Pilot subjects were run individually across two days. Upon arriving on Day One, the subject 
completed a pilot background questionnaire (providing such information as aircraft type ratings 
and years as a professional pilot) and signed an Informed Consent form, a Physiological 
Measures Informed Consent Form, and a High-Speed Research Program non-disclosure 
agreement. Subjects were then given a brief description of the experiment and its purpose. 

Training for the experiment then began. Subjects were brought to the Vertical Motion 
Simulator and were introduced to all aspects of the simulator flight deck’s operation. They first 
received training on the use of the side-arm controller, the XVS symbology and its use, flap 
settings, and engine controls. They were then introduced to special handling characteristics of 
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the HSCT aircraft model. Subjects were then permitted to fly four trials (two of each of the two 
training scenarios) with the conformal display. Upon completing these initial trials, the four 
conformality displays and conditions were described and subjects flew four trials (two of each of 
the two training scenarios) at each of the four conformality display conditions. 

Therefore, subjects flew a total of 20 training trials (ten of each of the two training scenarios), 
four initial trials with the conformal display and four at each of the four conformality display 
conditions. Upon completing these training trials, the experimenter answered the subject's 
questions and the subject was then given a ten-minute break while the VMS was configured for 
the first set of experimental trials. 

After the break, the heart rate monitor and EOG electrodes were attached and calibrated and 
subjects were then reseated to perform the Flight Path Control and Hazard Avoidance 
experimental trials. Half of the subjects performed the motion trials first, followed by the no 
motion trials while the other half of the subjects performed these trials in the reverse order. As 
noted earlier, within a block of motion or no motion trials, the trials were also blocked by 
conformality, such that subjects received a set of trials at one conformality display type. After 
completing each block of trials with one conformality display, subjects completed the TLX and 
usability questionnaire for that specific display. They then performed the next set of 
conformality display trials, followed by the TLX and usability questionnaire. This procedure 
continued until all experimental trials, TLX ratings, and usability questionnaires were completed 
within one motion/no motion block (resulting in a total of four TLX ratings and four usability 
questionnaires). 

After performing the first set of Flight Path Control and Hazard Avoidance experimental 
trials, the subject was given a ten-minute break during which the VMS was configured for the 
second set of experimental trials, then the experimental trials resumed. Prior to performing the 
second set of experimental trials, a single trial was given using the first training scenario. This 
was followed by the full set of remaining Flight Path Control and Hazard Avoidance 
experimental trials. Within this set of trials, subjects again completed the TLX and usability 
questionnaire after each set of trials at each of the four conformality displays (resulting in four 
additional completed TLX and usability questionnaires, for a total of eight of each type). The 
subject was then given a ten-minute break and the VMS was configured for the four Terrain 
Collision Avoidance trials. After completing the Terrain Collision Avoidance trials, subjects 
were asked to verbally rank ( 1 through 4) the four types of conformality displays. After the 
ranking, Day One data collection was completed. 

On Day Two, the subject was reminded of the experimental task and was given practice with 
two trials with the first training scenario. The subject then received the first set of Flight Path 
Control and Hazard Avoidance trials. Half of the subjects received the motion trials first, 
followed by the no motion trials and half the subjects received the trials in the reverse order. As 
during Day One, within each set of motion/no motion trials, the trials were blocked by 
conformality display condition. After each set of conformality displays, the subject again 
completed the NASA TLX and usability questionnaire. After performing all of the first set of 
experimental trials, the subject was then given a ten-minute break during which the VMS was 
configured for the remaining experimental trials. The subject then received the last set of 
experimental trials, again completing the NASA TLX and usability questionnaire after each set 
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of conformality display types. The subject was then debriefed, paid, and released for the day. 

Results 

Objective Data Analyses 

Analyses were conducted on the objective aircraft state and pilot subject state variables noted 
in Appendix B for both Flight Path Control and Hazard Avoidance performance evaluations 
(Scenarios 1 and 4). To maximize the sensitivity of these variables, many were analyzed for 
specific portions or times within the approach and landing task. One specific portion or time of 
interest was during turns, because turns involved looks from the forward display to the side 
window display and back, a time during which changes in display conformality might most 
affect task performance. Summaries of Analysis of Variance (ANOVA) statistical procedures 
are presented in Table 6 for turn 2 (downwind to base turn) and Table 7 for turn 3 (base to final 
turn). No significant differences were found in any of the task performance measures for either 
conformality condition or simulator motion or their interaction. Specific variables examined 
included altitude maintenance, bank angle standard deviation, maximum bank angle reached, and 
distance to target aircraft. In terms of objective measures of operator behavior, no significant 
differences were found for heart rate or for number of “looks” or transitions from the forward to 
side window displays. One exception was the “left looks” during turn 3 (base to final turn) 
where there was a significantly greater number of “left looks” for the motion condition (mean = 
18.94 transitions) compared to the no-motion condition (mean = 15.75 transitions), F( 1,5) = 
7.955, p<.05. However, there was no concomitant significant difference between the 
conformality conditions, and this difference attributed to motion was not reflected in any other 
measure. 
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Table 6. Summary of Analysis of Variance Results for Turn 2 (Downwind to Base) 


Measure 

Conformality 
Condition 
ANOVA F-value 
F<3,15) 

Conformality 

Condition 

ANOVA 

Probability value 

Motion 
Condition 
ANOVA F-value 
F(l,5) 

Motion Condition 
ANOVA 

Probability value 

Altitude Deviation 
Maximum 

.556 

.652 n.s. 


.062 n.s. 

Altitude Deviation 
Minimum 

.701 

.566 n.s. 

.380 

.565 n.s. 

Altitude Mean 

1.069 

.392 n.s. 

.045 

.841 n.s. 

Altitude Std. Dev. 

1.296 

.312 n.s. 

.792 

.414 n.s. 

Roll - Maximum 
Left 

2.585 

.092 n.s. 

.532 

.499 n.s. 

Roll - Maximum 
Right 

.552 

.654 n.s. 

.178 

,691 n.s. 

Roll - Left Mean 

2.148 

.137 n.s. 

.126 

.737 n.s. 

Roll Left Std. Dev. 

1.136 

.366 n.s. 

.162 

.704 n.s. 

Other Aircraft 
Minimum Distance 

.688 

.573 n.s. 

2.175 

.200 n.s. 

Left Looks (EOG) 

.593 

.629 n.s. 

2.796 

. 155 n.s. 

Heart Rate Mean 

.701 

.566 n.s. 

.493 

.514 n.s. 

Heart Rate Std. 

Dev. 

1.476 

.261 n.s. 

.531 

.499 n.s. 


** denotes p < .01. * denotes p<.05, n.s. denotes not significant (p >.05) 
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Table 7. Summary of Analysis of Variance Results for Turn 3 (Base to Final) 


Measure 

Conformality 
Condition 
ANOVA F-value 
F(345) 

Conformality 

Condition 

ANOVA 

Probability value 

Motion 
Condition 
ANOVA F-value 
F(l,5) 

Motion Condition 
ANOVA 

Probability value 

Altitude Deviation 
Maximum 

.996 

.422 n.s. 

1.018 

.359 n.s. 

Altitude Deviation 
Minimum 

1.089 

.384 n.s. 

2.041 

.212 n.s. 

Altitude Mean 

.834 

.496 n.s. 

2.215 

.197 n.s. 

Altitude Std. Dev, 

.485 

.687 n.s. 

.060 

.817 n.s. 

Roll - Maximum 
Left 

.549 

.656 n.s. 

2.279 

.192 n.s. 

Roll - Maximum 
Right 

.849 

.489 n.s. 

.493 

.514 n.s. 

Roll - Left Mean 

.726 

.552 n.s. 

4.413 

.090 n.s. 

Roll Left Std. Dev. 

.997 

.421 n.s. 

1.189 

.325 n.s. 

Other Aircraft 
Minimum Distance 

.125 

.944 n.s. 

2.806 

.155 n.s. 

Left Looks (EOG ) 

1.612 

.228 n.s. 

7.955 

.037 * 

Heart Rate Mean 

.630 

.607 n.s. 

.461 

.527 n.s. 

Heart Rate Std. 
Dev. 

1.221 

.336 n.s. 

.055 

.824 n.s. 


** denotes p < .01, * denotes p<.05, n.s. denotes not significant (p >.05) 


Approach and Landing 

Analyses of flight technical error parameters were conducted at 1000 feet before runway 
threshold crossing, at 500 before threshold, and at threshold. A summary of ANOVA procedures 
conducted at these points is presented in Table 8. No significant differences were found for 
conformality or motion condition for altitude, sink rate (HDOT), or airspeed at any of the 
selected points. The landing touchdown simulated aircraft performance data ANOVA 
procedures are summarized in Table 9. The landing “touchdown” performance data showed no 
significant differences in touchdown sink rate. However, the touchdown point in the non- 
conformal conditions was found to be significantly further down the runway, F(3,15) = 4.957, 
p<.02, and at a significantly slower airspeed, F(3,15) = 13.109, p<.001. In addition, there was a 
significant motion/no motion effect for the touchdown airspeed, F( 1,5) = 15.409, p<.02, with 
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slower airspeeds in the motion condition. This finding most likely represents a strategy chosen 
by the test subjects to touch down gently in the motion condition, which led to slower airspeed 
and, therefore, longer touchdown position, in order to avoid the pronounced bang/bump of the 
simulator at touchdown. A significant lateral position finding (Table 9) is probably not of 
practical significance as the mean difference was very small (approximately 5 feet). 


Table 8. Summary of Analysis of Variance Results at selected points on final approach 


Measure 

Conformality 
Condition 
ANOVA F-value 
F(3,15) 

Conformality 

Condition 

ANOVA 

Probability value 

Motion 
Condition 
ANOVA F-value 
F(l,5) 

Motion Condition 
ANOVA 

Probability value 

Altitude 1000 ft 
before runway 
threshold 

.343 

.794 n.s. 

.524 

.502 n.s. 

Vertical Rate 
(HDOT) 1000 ft 
before runway 
threshold 

.111 

.953 n.s. 

2.049 

.212 n.s. 

Altitude 500 ft 
before runway 
threshold 

.864 

.481 n.s. 

.164 

.702 n.s. 

Vertical Rate 
(HDOT) 500 ft 
before runway 
threshold 

.789 

.519 n.s. 

1.164 

.330 n.s. 

Altitude at runway 
threshold 

2.253 

.124 n.s. 

.006 

.942 n.s. 

Vertical Rate 
(HDOT) at runway 
threshold 

1.298 

.312 n.s. 

.479 

.520 n.s. 


** denotes p < .01, * denotes pc. 05, n.s. denotes not significant (p >.05) 


Table 9. Summary of Analysis of Variance Results at Runway Touchdown 


Measure 

Conformality 
Condition 
ANOVA F- value 
F(3,15) 

Conformality 

Condition 

ANOVA 

Probability value 

Motion 
Condition 
ANOVA F-value 
F(l,5) 

Motion Condition 
ANOVA 

Probability value 

Longitudinal 
Runway Position 

4.957 

.014* 

4.107 

.099 n.s. 

Lateral Runway 
Position 

6.111 

.006 ** 

.098 

.767 n.s. 

Vertical Rate 
(HDOT) 

.173 

.913 n.s. 

.576 

.482 n.s. 

Airspeed 

13.109 

.001 ** 

15.409 

.011 * 


** denotes p < .01 , * denotes pc. 05, n.s. denotes not significant (p >.05) 
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Hazard avoidance 


The hazard avoidance scenarios (Scenarios 2, 3, and 5) were examined to determine if there 
were performance differences due to conformality condition, simulator motion, or their 
interaction. No significant differences in any of the parameters were noted for any of these 
scenarios. No significant differences were found in timing of avoidance maneuvers. Likewise, 
no significant differences in test subject heart rate or “looks” to the side window were found for 
these scenarios. 

Mountain Scenario/Terrain Collision Avoidance 

The mountain scenarios (Scenario 6) were all flown with motion on. No significant 
differences were found for the conformality condition in any of the flight technical or test subject 
objective behavior measures. 

Subjective Data Analyses 

Three subjective measures were taken during experimental trials: NASA TLX Workload 
Ratings, preference rankings of the (conformal and nonconformal) displays, and questionnaire 
ratings of display usability. 

NASA TLX Workload Ratings 

The NASA TLX scale (ref. 10) is given in Appendix D. This instrument is widely used to 
measure subjective evaluations of perceived workload and fatigue. There are six 
workload/fatigue categories evaluated by the subject: Mental, Physical, Temporal, Performance, 
Effort, and Frustration Level. Each category is associated with a ten-point scale, with zero 
considered the lowest value (e.g., little or no mental effort is required to perform the task) and 
100 the highest value on that scale (e.g., maximum mental effort is required). 

The TLX ratings from all subjects were combined and mean values were calculated for each 
of the six scales for motion/no motion conditions and across the four conformality display 
conditions for both Scenarios 1 and 4. These data are given in Appendix E and are summarized 
in Table 10 (for Scenario 1 ) and Table 1 1 (for Scenario 4). 

Mean TLX ratings were compared by Analysis of Variance and showed no significant 
differences across main effect conditions of motion, scenario, or conformality and no significant 
interactions. This indicated that, as a group, the test subjects were not reporting significant 
differences in perceived workload or fatigue across the conditions under evaluation. It should 
also be noted that no high (>75) mean workload ratings were obtained in the test sessions. 
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Table 10. Mean Pilot Subjective Ratings of Perceived Workload for Scenario 1 
Motion and No Motion Conditions 
(NASA TLX Scale) 

MOTION 



Mental 

Physical 

Temporal 

Perfor- 

mance 

Effort 

Frustration 

Level 

Means 

Conformal 

40.8 

38.3 

40.0 

30.8 

37.5 

17.5 

34.15 

4 Degrees 

39.0 

39.0 

42.0 

33.0 

43.0 

20.0 

36.00 

8 Degrees 

41.7 

38.3 

32.5 

25.0 

38.3 

20.8 

32.77 

8+Degrees 

47.5 

44.2 

35.8 

25.0 

52.5 

20.0 

37.50 


NO MOTION 



Mental 

Physical 

Temporal 

Perfor- 

mance 

Effort 

Frustration 

Level 

Means 

Conformal 

49.2 

41.7 

25 

24.2 

46.7 

14.2 

33.50 

4 Degrees 

42.5 

40.8 

30.8 

30.8 

46.7 

19.2 

35.13 

8 Degrees 

40.8 

41.7 

31.7 

26.7 

44.2 

20 

34.18 

8+Degrees 

41.7 

33.3 

29.2 

19.2 

38.3 

14.2 

29.32 


Table 1 1. Mean Pilot Subjective Ratings of Perceived Workload for Scenario 4 Motion and No 

Motion Conditions (NASA TLX Scale) 

MOTION 



Mental 

Physical 

Temporal 

Performance 

Effort 

Frustration 

Level 

Means 

Conformal 

42.5 

42.5 

41.3 

33.8 

50 

25 

39.18 

4 Degrees 

41 

37 

38 

29 

35 

20 

33.33 

8 Degrees 

30 

37 

32 

34 

28 

24 

30.83 

8+Degrees 

40 

34 

28 

36 

29 

22 

31.50 


NO MOTION 



Mental 

Physical 

Temporal 

Performance 

Effort 

Frustration 

Level 

Means 

Conformal 

32.5 

30 

27.5 

20 

22.5 

13.8 

24.38 

4 Degrees 

41 

38 

32 

26 

39 

26 

33.67 

8 Degrees 

39 

36 

32 

29 

32 

24 

32.00 

8+Degrees 

39.2 

35 

31 

28 

34 

22 

31.53 
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Conformality/Nonconformality Display Preference Rankings 

Subjects were also asked to rank order the four conformal/nonconformal displays for 
preference (with one the “most preferred” and four the “least preferred” display). These 
rankings are summarized in Table 12. All subjects except one preferred the conformal display. 
Verbal comments from the one pilot preferring the 8 degree non-conformal (22 degree total 
VFOV) display indicated that he may have been responding to a preference for a greater vertical 
field of view (relative to the 14 degree field used in all of the other conditions) rather than to 
conformality. As indicated by the means, the preference trend across all subjects was: conformal 
display most preferred, the 8+ degree display (with the wider field of view) next preferred, and 
the two nonconformal shortened field of view displays (4 and 8 degrees) least preferred. 

Table 12. Pilot Preference Rankings of Conformal/nonconformal displays 


Pilot 

Conformal 

4 Degree 

8 Degree 

8 Degree+ 

1 

1 

3 

4 

2 

? 

1 

4 

2 

3 

3 

1 

2 

4 

3 

4 

1 

2 

4 

3 

5 

3 

4 

2 

1 

6 

1 

4 

3 

2 

Mean Rankings 

1.33 

3.17 

3.17 

2.33 


Conformality /Nonconformality Display Usability Ratings 

A questionnaire was developed to assess the perceived “usability” of the conformal/ 
nonconformal displays. Subjects were asked to assess each display on a six-interval scale. The 
scale was structured such that the low anchor was “Much Less Effort Required” and the upper 
anchor was “Much More Effort Required,” with a neutral anchor, “No More / No Less Effort 
Required.” Subjects indicated each display's “usability” by selecting a rating on the scale with 
regard to a series of five questions: 

( 1 ) Usability of this configuration for maintaining level flight. 

(2) Usability of this configuration for scanning for traffic. 

(3) Usability of this configuration for landing approach. 

(4) Usability of this configuration for landing flare 

(5) Usability of this configuration for nose gear touchdown (de-rotation). 

These data are provided in Appendix F and the means are summarized in Table 13 (by motion 
condition and display type for Scenarios 1 and 4). No trends or special results were found with 
these ratings. The results generally followed the trends indicated with the display preference 
ratings, i.e., generally the conformal and 8+ degree displays were preferred. 
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Table 13. Mean Pilot Ratings of Conformal & Nonconformal Displays 
as a Function of Motion Condition and Scenario 
( 1 = “less effort"; 6 = “more effort”) 


Condition 

Conformal 

4 Degree 

8 Degree 

8 Degree+ 






Scenario 1 





Motion 

4.3 

4.0 

4.5 

4.1 

No Motion 

3.3 

3.3 

3.3 

2.9 






Scenario 4 





Motion 

2.9 

3.4 

3.5 

3.3 

No Motion 

3.1 

3.2 

3.6 

3.4 


Pilot Comments 

While completing the questionnaire, subjects were permitted to annotate their ratings with 
comments. The majority of comments related to the design or usability of the XVS symbology, 
display clutter, “blind spots,” and difficulties with the VMS structure occluding parts of the 
visual field. The pilot comment data were not statistically analyzed, but were collected and 
organized for readability. All pilot comments made on the questionnaires are provided for 
reference in Appendix G. 

General Conclusions and Recommendations 

This experiment is one of a series of experiments exploring the “design space” restrictions for 
placement of an XVS display. In this study, the primary experimental issue examined was 
conformality of the forward display vertical position with respect to the side window in 
simulated flight conditions. In particular, this study quantified the effects of visual conformality 
on pilot flight path control and hazard avoidance performance. For this study, conformality 
related to the positioning and relationship of the horizon line and associated symbology 
presented on the forward display and the horizon and associated ground, horizon, and sky 
textures presented in the side window display. The side window, though computer generated, 
represented a real window and a view of the real world. For this experiment the forward display 
symbology was presented as conformal or shifted up by 4 or 8 degrees visual angle. This shift 
was incorporated to permit the desired amount of below-horizon look-down angle on the forward 
displays on approach for the high angle of attack HSCT. 

With regard to the three primary hypotheses under test in the present study, the results were as 
follows: ( 1 ) There was no evidence to support the first hypothesis that flight control 
performance and airborne hazard detection would be degraded in the nonconformal conditions. 
No operationally significant display conformality effects were noted in terms of simulated 
aircraft performance measures during the approach phase, turns or during hazard avoidance 
scenarios. An effect on runway “touchdown” point and airspeed was found and an explanation 
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was noted for this finding. (2) The second hypothesis was supported in that no significant 
differences were found between motion and no-motion throughout the tests except for 
touchdown “footprint’' location and airspeed which may have been a function of performance of 
the VMS simulator (discussed below). (3) While pilots typically express a preference for 
conformal displays, the general feeling was that the changes in VFOV could be adapted to 
through training. The overall restriction in VFOV angle used in this experiment may have been 
a larger factor in acceptability and use than the shift in VFOV. 

The findings of this and related conformality experiments may impact the location and 
positioning of displays and sensors driving the displays in the HSCT. The results of this study 
indicate that VFOV displacements of 4 or 8 degrees in the forward display may be accomplished 
with minimal consequences on performance. No cases of simulator sickness or other physical 
signs of motion discomfort or vestibular effects were reported by the test subjects for any of the 
display conditions. 

Touchdown and Flare 

No significant differences were observed in positional or rate measures at 1000 feet before 
runway threshold, or 500 feet before threshold, or at runway threshold. The landing 
“touchdown” (defined by main gear touchdown) performance data showed no significant 
differences in touchdown sink rate. However, the touchdown point in the non-conformal 
conditions was found to be significantly further down the runway and the touchdown was 
accomplished at a significantly lower airspeed. These findings may reflect uncertainty about 
flaring the aircraft when there is a large amount of forward scene and side window mismatch. 
This finding most likely represents a strategy chosen by the test subjects to touch down gently in 
the motion condition, which led to longer touchdown position and slower airspeed in order to 
avoid the pronounced bang/bump of the simulator at touchdown. Pilot comments indicated that 
non-conformal VFOV displays for flaring the aircraft were different but could be adapted to 
through training. 

Turns when following a lead aircraft 

In the scenarios that required following traffic on downwind, base, and final, specific 
attention was paid to the data from the downwind to base and base to final turns, as these were 
times when the lead aircraft would transition from the forward display to the side window and 
back to the forward display. No significant differences were observed in any of the aircraft 
position or position deviation data across the conformality conditions. Likewise, no significance 
by conformality condition was noted in the EOG-based measure of the number of looks from the 
forward display to the side window. 

It should be noted that all bank angles were relatively shallow, perhaps a function of the 
limited vertical field of view in which it was easy to decrease the horizon in view when it was 
occluded by both the top and bottom of the display. In addition, the horizon could easily move 
up and out of the left side window in a left turn, meaning that only ground could be seen because 
of the limited field of view of the side window display. 
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Hazard avoidance 


Examination of the set of data from the hazard avoidance scenarios showed that the 
conformality factor had no effect on performance in these scenarios. Specific performance 
indicators examined included: time of beginning avoidance maneuver, pitch and roll angles 
(both sustained and maximum angles reached), aircraft position and airspeed, heading and 
altitude errors, distance from target/hazard, and frequency of left window looks (an EOG-based 
measure). Narrative remarks during the mountain scenarios indicated that non-conformality was 
most noticeable in those scenarios. This may be due to the fact that a fixed or level horizon was 
no longer present in the side window (or front scene, other than XVS symbology). Remarks also 
noted the increased value of the XVS horizon line and flight path vector when an irregular 
horizon, such as from mountain peaks or ridge lines, is present. 

Subjective Ranking 

Subjective rankings of the display conditions revealed that five of the six pilots preferred the 
conformal display. Verbal comments from the one pilot preferring the 8-degree non-conformal 
(22 degree total VFOV) display indicated that he may have been responding to a preference for a 
greater VFOV (relative to the 14 degree field used in all of the other conditions) than to the 
conformality issue. The VFOV was masked in three of the four conditions in order to maintain 
the same amount of "lookdown” angle for each of the conformality conditions. 

Lessons Learned 

It is possible that the effects of non-conformality may have been understated in the present 
simulator experiment due to the wide pillar between the front display and the side window 
(approximately 12 to 15 degrees visual angle) in the VMS facility. In addition, the effect on 
landing flare of the limited VFOV used in this study is unknown and may have led to the longer 
touchdown distances that were observed. 

Another limitation concerns the hazards employed in the present study. Replications of 
experimental runs meant that test subjects saw similar hazards more than once. Obviously, the 
hazard could only be a “surprise” once, so it would be recommended that future studies 
incorporate hazards that are unpredictable in timing and/or location. Additional 
recommendations include: ( 1 ) the future use of “breakout” scenarios to assess display use in low 
visibility conditions; (2) increased subject pilot workload through “full mission” approaches, 
including events and activities such as ATC calls, crew interactions, last minute runway changes, 
weather at minimums, cross winds, cockpit interruptions; and (3) increased sample size to reduce 
overall between-subject variability. 

Summary 

The incongruities in visual cues associated with nonconformality in vertical display location 
had minimal performance consequences on a simulated approach and landing task based on 
testing of six pilots in the NASA Langley VMS. Performance effects observed were noted 
above. No cases of simulator sickness or other physical signs of motion discomfort or vestibular 
effects were reported for any of the display conditions. When asked to rank the display 
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conditions, five out of six pilot subjects preferred the conformal display condition. 

Factors to consider in interpreting these findings include: ( 1 ) the limited VFOV of the XVS 
display evaluated here due to hardware and experimental constraints, (2) the wide pillar 
separating the forward display and the side window (about 12 to 15 degrees), (3) the effect of 
limited VFOV on bank angle, (4) all hazards used in the experiment could be described as “slow 
onset,” meaning that sudden, immediate, and large control inputs were not required for hazard 
avoidance, and (5) all hazard scenarios incorporated good visibility (Visual Meteorological 
Conditions, or VMC). It would be valuable to evaluate conformality effects on hazard detection 
and avoidance in break-out conditions or other situations where a rapid “mental reconstruction” 
of the visual scene or situation was required. 

While the High Speed Research (HSR) program was cancelled prior to industry construction 
of a prototype aircraft using the display concepts under test here, these results have implications 
beyond the HSR program. These findings are important for activities of the Synthetic Vision 
Systems element of the Aviation Safety Program which faces issues of non-conformality in 
terms of Head-Down Display field-of-view and display scene “minification”. Additionally, 
these findings have implications for display design in general and as a stepping stone for future 
efforts in forward displays for High Speed Aircraft on which a no-droop nose concept is chosen. 
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Appendix A 

Summarized Pilot Subjects Information 
I. Pilot Experience 


Subject 

Years Flying 
Commercial 

Years Flying 
Military 

Total Hours 
Flying 

Total Hours 
As Pilot-in- 
Command 

Glass Cockpit 
Experience 
(years) 

1. F/O 

6 

10 

5348 

2600 

1-5 

2. Capt 

18 

— 

17,000+ 

15,000+ 

0 

3. F/O 

8 

21 

10,000 

4,000 

6 

4. F/O 

4.5 

20 

5,700 

3,200 

1-5 

5. F/O 

7 

— 

3,700 

3,200 

0 

6. F/O 

15 

— 

9,500 

6,500 

1-5 

Means 

9.75 Years 

17 Years 

8,541 Hours 

5,300 Hours 

0 years N=2 

1 -5 years N=3 
6+years N= 1 


II. Aircraft Types Flown by Pilot Subjects 


• B727, B 7 3 7 - 200/300/400, B747, B757, B767 

• DC-9, MD-80 

• T-37, T-38, T-39, T-44 

• F-100, F15 

• E- 1 , E2C 

« DHC-7, DHC-8 

• SD-330 

• B-99 

• Gulfstream IV 

• Grumman GA-7 
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Appendix B 

Objective & Subjective Dependent Measures 


OBJECTIVE MEASURES of AIRCRAFT STATE 


1 Hz data rate from start to end of run 
Pilot, Run, and Scenario number 
Display conformality condition (0°, 4°, 8°, 8°+) 

Elapsed time 

Aircraft position in X, Y, and Z 
Aircraft attitude: 

psi (heading) 
phi (roll) 
theta (pitch) 

Aircraft indicated airspeed 

Target aircraft position (Xt, Yt, Zt) (for all targets) 

Distance to target aircraft (for all targets) 


20 Hz data rate from five nm to touchdown 
Pilot, Run, and Scenario number 
Display conformality condition (0°, 4°, 8°, 8°+) 

Elapsed time 
Lateral path error (feet) 

Vertical path error (feet) 

Pitch, Roll, and Sink rates 
Side-arm controller pitch input angle 
Side-arm controller roll input angle 
Rudder peddal position 
Throttle position 
Engine power (%) 


Single event data upon touchdown 
Pilot, Run, and Scenario number 
Display conformality condition (0°, 4°, 8°, 8°+) 

At first gear touchdown: 

Aircraft position (X, Y) 

Elapsed time 
Sink rate 
Lateral velocity 
Indicated airspeed 
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At nosewheel touchdown: 

Aircraft position (X, Y) 
Elapsed time 
Lateral velocity 
Pitch rate 


OBJECTIVE MEASURES of PILOT SUBJECT STATE 

Heart rate (measured via earclip plethysmograph) 

EOG (electro-oculogram to detect “left looks") 


SUBJECTIVE MEASURES 

NASA TLX Workload Rating Scale 

Usability Questionnaire concerning display conformality 

Pilot comments 
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Appendix C 

Example Pilot Subject Running Order 
DAY ONE 


Training Experiment 2 

(All No Motion Trials) No Motion Trials Motion Trials 


Scenario 

Fwd Angle 

Scenario 

Fwd Angle 

Scenario 

Fwd Angle 





T1 

4 

T1 

0 

4 

8 

4 

4 

T1 

0 

4 

8 

5 

4 

T2 

0 

4 

8 

4 

4 

T2 

0 

4 

4 

4 

4 



4 

4 

5 

4 

T1 

0 

4 

4 

5 

4 

T1 

0 

4 

0 

5 

8+ 

T2 

0 

4 

0 

4 

8+ 

T2 

0 

4 

0 

5 

8+ 

T1 

4 

4 

8+ 

4 

8+ 

T1 

4 

4 

8+ 

4 

8+ 

T2 

4 

4 

8+ 

5 

8 

T2 

4 



4 

8 

T1 

8 



4 

8 

T1 

8 



5 

8 

T2 

8 



4 

8 

T2 

8 



4 

0 

T1 

8+ 



5 

0 

T1 

8+ 



4 

0 

T2 

8+ 



4 

0 

T2 

8+ 



5 

0 

Total 

20 

Total 

12 

Total 

21 


Terrain Collision Avoidance Trials 





(All with Motion) 





Scenario Fwd Angle 





6 

8+ 





6 

4 





6 

0 





6 

8 





Total 

4 





C-l 





Total 33 Total 


CO 


ro co — l ro 


co ro 


co ro 


co ro 


roco-^rocoro-^co-^coro-^ 


H H 


c n 
n 

Q 

3 


OOOOOOOOODOOOO^-^^^^-U^OOOOOOOOOOOOOOOOOOOOOOOCD 

+ + + + + + + + + 


00 00 
+ + 


a 

> 

3 

CD 


o 

<D 

3 

to 


ro 


^^^ooooooooooooooo 

+ 4 - + 


s 

a 


<o 

o 


Motion Trials No Motion Trials 



Appendix D 

NASA TLX Workload Rating Scale 


Title & Description 

MENTAL DEMAND: 

How much mental and perceptual activity was required (e.g., thinking, deciding, calculating, 
remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, 
exacting or forgiving? 

Mental Demand 


Low High 


PHYSICAL DEMAND: 

How much physical activity was required (e.g., pushing, pulling, turning, controlling, activating, 
etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious? 

Physical Demand 


Low High 


TEMPORAL DEMAND: 

How much time pressure did you feel due to the rate or pace at which the tasks or task elements 
occurred? Was the pace slow and leisurely or rapid and frantic? 

Temporal Demand 


Low High 


D-l 


PERFORMANCE: 

How successful do you think you were in accomplishing the goals of the task set by the 
experimenter (or yourself)? How satisfied were you with your performance in accomplishing 
these goals? 

Performance 



Good 


Poor 


EFFORT: 

How hard did you have to work (mentally and physically) to accomplish your level of 
performance? 


Effort 



Low High 


FRUSTRATION LEVEL: 

How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, 
relaxed and complacent did you feel during the task? 


Frustration 



Low High 
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Means 41.7 33.3 29.2 19.2 38.3 14.2 
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Summan of NASA TLX Ratings for Scenario 1 and Scenario 4 
Motion and No Motion Conditions across Conformality Displays 



Conformal 

Means 
4 Degrees 

Means 
8 Degrees 

Means 
8 Degrees + 

Means 


Mental 

Physical 

60 

65 

45 

45 

20 

10 

35 

35 

75 

65 

10 

10 

40.8 

38.3 

(No Subj 1 ) 

(No Subj 1 ) 

45 

45 

10 

10 

45 

45 

80 

80 

J5 

15 

39 

39 

70 

75 

55 

35 

10 

10 

45 

45 

60 

55 

10 

10 

41.7 

38.3 

75 

70 

65 

65 

20 

10 

45 

45 

70 

65 

H) 

10 

47.5 

44.2 


Scenario 1 


Motion 

Temporal 

Performance 

55 

15 

60 

60 

20 

15 

35 

50 

55 

35 

15 

10 

40 

30.8 

(No Subj 1 ) 

(No Subj 1 ) 

45 

50 

10 

10 

45 

50 

70 

35 

40 

20 

42 

33 

50 

10 

35 

50 

10 

10 

45 

50 

50 

20 

5 

10 

32.5 

25 

60 

20 

50 

40 

10 

20 

45 

50 

45 

10 

5 

10 

35.8 

25 


Effort 

Frustration Lvl 

60 

10 

65 

40 

10 

5 

55 

25 

30 

25 

5 

0 

37.5 

17.5 

(No Subj 1 ) 

(No Subj 1 ) 

40 

35 

10 

0 

65 

35 

80 

20 

20 

10 

43 

20 

75 

10 

30 

55 

10 

0 

70 

35 

45 

25 

0 

0 

38.3 

20.8 

65 

10 

70 

50 

20 

10 

75 

35 

75 

15 

JO 

0 

52.5 

20 
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Scenario 4 
No Motion 


Conformal 

Means 
4 Degrees 

Means 
8 Degrees 

Means 
8 Degrees + 


Physical Temporal 


Effort Frustration Lvl 


Mental 
(No Subj 1 ) 
(No Subj 2) 
10 
50 
60 
10 
32.5 

(No Subj 1 ) 
35 
20 
50 
80 
20 
41 

(No Subj 1 ) 
45 
20 
50 
60 
20 
39 

(No Subj 1) 
56 

10 

50 

60 

20 

39.2 


(No Subj 1 ) 

(No Subj 2) 
10 
50 
50 
10 
30 

(No Subj 1) 

35 
20 
50 
65 
20 
38 

(No Subj 1 ) 
45 
20 
50 
45 
20 

36 

(No Subj 1 ) 
45 
10 
50 
50 
20 
35 


(No Subj 1 ) 
(No Subj 2) 

10 

45 

45 

JO 

27.5 

(No Subj 1) 
35 
20 
50 
45 
JO 
32 

(No Subj 1 ) 
45 
20 
50 
35 
JO 
32 

(No Subj 1) 
45 
10 
50 
40 
JO 
31 


Performance 

(No Subj 1 ) 

(No Subj 2) 
10 
50 
10 
JO 
20 

(No Subj 1 ) 
35 
20 
50 
15 
JO 
26 

(No Subj 1) 
55 
20 
50 
10 
JO 
29 

(No Subj 1) 
50 
10 
50 
20 
JO 
28 


(No Subj 1 ) 
(No Subj 2) 
10 
55 
25 
0 

22.5 

(No Subj 1) 
35 
20 
70 
70 
0 

39 

(No Subj 1) 
50 
20 
65 
15 
JO 
32 

(No Subj 1 ) 
45 
10 
60 
55 
0 

34 


(No Subj 1 ) 
(No Subj 2) 
0 
45 
10 
0 

13.8 

(No Subj 1) 
35 
10 
65 
20 
0 

26 

(No Subj 1 ) 
40 

10 

60 

10 

0 

24 

(No Subj 1) 
55 
0 

40 

15 

0 

22 


Means 



Conformal 

Means 
4 Degrees 

Means 
8 Degrees 

Means 
8 Degrees + 

Means 


Scenario 4 
Motion 


Mental 

Physical 

Temporal 

Performance 

Effort 

Frustration Lvl 

( No Subj 1 ) 

(No Subj 1) 

(No Subj 1 ) 

(No Subj 1) 

(No Subj I ) 

( No Subj 1 ) 

60 

60 

60 

40 

65 

35 

5 

5 

10 

10 

10 

0 

25 

25 

25 

50 

50 

25 

80 

80 

70 

35 

75 

40 

(No Subj 6) 

(No Subj 6) 

( No Subi 6) 

(No Subi 6) 

(No Subi 6) 

(No Subi 6) 

42.5 

42.5 

41.3 

33.8 

50 

25 

(No Subj I ) 

( No Subj 1 ) 

(No Subj 1 ) 

(No Subj 1) 

(No Subj 1 ) 

(No Subj 1 ) 

35 

25 

25 

25 

25 

25 

10 

10 

20 

10 

10 

0 

50 

50 

50 

50 

60 

40 

80 

80 

75 

40 

70 

35 

30 

20 

20 

20 

K) 

0 

41 

37 

38 

29 

35 

20 

(No Subj 1 ) 

(No Subj 1) 

(No Subj 1) 

(No Subj 1 ) 

(No Subj 1 ) 

(No Subj 1) 

20 

50 

40 

60 

45 

70 

0 

5 

10 

10 

10 

0 

25 

25 

25 

50 

40 

35 

75 

75 

65 

20 

25 

15 

30 

30 

20 

30 

20 

0 

30 

37 

32 

34 

28 

24 

(No Subj 1) 

(No Subj 1) 

(No Subj 1) 

(No Subj I ) 

(No Subj 1 ) 

(No Subj 1) 

45 

35 

35 

35 

35 

30 

5 

5 

5 

10 

10 

0 

45 

45 

50 

50 

55 

45 

70 

55 

45 

15 

25 

25 

35 

30 

5 

70 

20 

10 

40 

34 

28 

36 

29 

22 
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Appendix F 

Summary of Display Usability Questionnaire Ratings 
(1 = “Much Less Effort Required” and 6 = “Much More Effort Required”) 

Q1 . Usability of this configuration for maintaining level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Q3. Usability of this configuration for landing approach. 

Q4. Usability of this configuration for landing flare 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Scenario 1 
No Motion 



Question 

Conf 

4 deg 

8 deg 

8+ deg 

Ql. 

Usability of this configuration 
for maintaining level flight. 

3.1 

3 

3.1 

2.8 

Q2. 

Usability of this configuration 
for scanning for traffic. 

3.1 

3.1 

2.8 

2.5 

Q3. 

Usability of this configuration 
for landing approach. 

3.5 

3.3 

3.3 

2.8 

Q4. 

Usability of this configuration 
for landing flare 

3.5 

3.7 

3.5 

3.3 

Q5. 

Usability of this configuration for 
nose gear touchdown (de-rotation). 

3.4 

3.4 

3.7 

3.3 


Mean Rating 

3.3 

Scenario 1 
Motion 

3.3 

3.3 

2.9 


Question 

Conf 

4 deg 

8 deg 

8+ deg 

Ql. 

Usability of this configuration 
for maintaining level flight. 

4.3 

4.1 

4.1 

4.3 

Q2. 

Usability of this configuration 
for scanning for traffic. 

4.3 

4 

4.6 

4 

Q3. 

Usability of this configuration 
for landing approach. 

4.5 

4 

4.5 

4 

Q4. 

Usability of this configuration 
for landing flare 

4.2 

3.7 

4.6 

4.2 

Q5. 

Usability of this configuration for 
nose gear touchdown (de-rotation). 

4.1 

3.7 

4.5 

4.3 


Mean Rating 

4.3 

4 

4.5 

4.1 
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Scenario 4 
No Motion 



Question 

Conf 

4 deg 

8 deg 

8+ deg 

Ql. 

Usability of this configuration 
for maintaining level flight. 

2.8 

2.9 

3 

3.2 

Q2. 

Usability of this configuration 
for scanning for traffic. 

3.2 

3.5 

3.2 

3.2 

Q3. 

Usability of this configuration 
for landing approach. 

3.2 

3.2 

3.6 

3.4 

Q4. 

Usability of this configuration 
for landing flare 

3.2 

3.1 

4.2 

3.6 

Q5. 

Usability of this configuration for 
nose gear touchdown (de-rotation). 

3.4 

3.4 

3.7 

3.3 


Mean Rating 

3.1 

Scenario 4 
Motion 

3.2 

3.6 

3.4 


Question 

Conf 

4 deg 

8 deg 

8+ deg 

Ql- 

Usability of this configuration 
for maintaining level flight. 

2.8 

3.2 

3.3 

3.1 

Q2. 

Usability of this configuration 
for scanning for traffic. 

3.3 

3.4 

3.4 

3.3 

Q3. 

Usability of this configuration 
for landing approach. 

2.8 

3.3 

3.3 

3.1 

Q4. 

Usability of this configuration 
for landing flare 

2.8 

3.4 

3.8 

3.6 

Q5- 

Usability of this configuration for 
nose gear touchdown (de-rotation). 

3 

3.5 

3.7 

3.6 


Mean Ratings 2.9 3.4 3.5 3.3 
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Appendix G 
Pilot Comments 

(Organized by Scenario, Motion/No Motion, and Question) 

The following are transcribed responses of handwritten comments in the blank space provided 
following each of the rating scale questions noted below. 

Scenarios 7, 2, and. 3 No Motion : 

• Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Felt better about aircraft ahead and below me without mask 
Except for blind spots 

Q3. Usability of this configuration for landing approach. 

Glidepath adjustment was easy and more normal looking 

Q4. Usability of this configuration for landing flare 

This configuration was very easy (in landing flare). 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Found myself varying the flight path to keep traffic in sight. 

General: When would you expect problems with this display configuration: 

This configuration looked more normal as 1 perceived it. 

No problems. 

"Blind area" between front and side view is distracting. [Resulting from the simulator.] 

• 4 degree N on-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

HUD easy to use. 

Q2. Usability of this configuration for scanning for traffic. 

Except for blind spots. 

Less effort in this case for following the traffic because of the wide view of the horizon. 
Q3. Usability of this configuration for landing approach. 

Had little problem ?? on last 200 feet. Adjusted glidepath and everything okay. 

Better view of ground references. 

Q4. Usability of this configuration for landing flare 
Last configuration was a little easier to use. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Appeared to carry a little less nose high attitude at touchdown. 

General: When would you expect problems with this display configuration: 

None except previously noted airspeed display partially cut out by upper mask. 

Blind spots for scanning bothered me a little. Adjustment for glide path took two 
approaches to cross threshold at 50 feet. 

HUD display too big for instrument flying. Needs to have pitch references at least every 
2.5 degrees. 

With top and bottom masking it would be more difficult to see traffic, both during high 
angles of attack and maneuvering in dense terminal areas with traffic above. 

Maybe looking for traffic above flight path. 
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• 8 degree Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Harder to maintain level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Same except for blindspots. 

With masking on the bottom, tends to have too much data cluttered in vertical field of 
view. The display seems to have no effect on this. 

Q3. Usability of this configuration for landing approach. 

Required more effort to establish glide path. 

Difficult to see target from 6 o'clock. 

Better in this case for traffic below flight path. 

Q4. Usability of this configuration for landing flare 
Masked top cuts off some symbology and degrades horizon cues. 

I might have been flaring too early on this one. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Too much nose up attitude which was difficult to correct for. 

General: When would you expect problems with this display configuration: 

During climbout with a high angle of attack and traffic crossing below. 

May not see traffic above flight path. 

• 8 degree + Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Easy to maintain. 

Q2. Usability of this configuration for scanning for traffic. 

Except for blind spots. 

Even with the non-conformal view the traffic is quite predictable as to where it will be 
when it moves from front screen to side window. 

Best display in this case (traffic below flight path). 

Q3. Usability of this configuration for landing approach. 

Glideslope and threshold were easy to judge. 

Best in this case (ground reference used to fly approach). 

Q4. Usability of this configuration for landing flare 
Much better than masked displays. 

Easy to judge flare point. 

Full view no mask tends to fly a 3 degree glideslope rather than a flat 2 1/2 degree 
approach. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

OK. 

General: When would you expect problems with this display configuration: 

Blind spots — panoramic view would make everything easier. 

No problems expected. 

May be distracting when HUD not used. May be hard to see traffic above flight path. 
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Scenarios 7, 2, and 3 Motion: 

• Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Fairly easy to stay level. 

Q2. Usability of this configuration for scanning for traffic. 

Surprised to pick traffic out side window so easily. 

Many blind spots. 

You cannot see enough above you, especially during turns in excess of 20 degees bank. 

Q3. Usability of this configuration for landing approach. 

Felt comfortable, easy to gauge glide path. 

In fact, I feel that the masked top creates a higher than actual altitude, leading to a low 
flat approach. 

Q4. Usability of this configuration for landing flare 

Comfortable with flare in this configuration. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Average. 

General: When would you expect problems with this display configuration: 

Blind areas. 

During extensive maneuvering in a highly congested terminal area, not enough vertical 
picture. 

• 4 degrees Non-Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Same in all configurations. 

Traffic hard to see on final approach. 

Vertical field of view still slightly limited. 

Limited vertical field of vision. 

Q3. Usability of this configuration for landing approach. 

Easier than last configuration (Subject 2). 

The limited view tends to make me fly a flat approach. 

Q4. Usability of this configuration for landing flare 
Easier to judge flare point and crossing threshold at 50 feet. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Same as before. 

General: When would you expect problems with this display configuration: 

Blind spots. Hard to see traffic on final. 

• 8 degrees Non-Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Less effort to maintain than last configuration (Subj 1 ). 

Q2. Usability of this configuration for scanning for traffic. 

Same as others. 

Did not initially see second aircraft — observed him on following runs. 
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With the HUD not decluttered, the traffic sometimes becomes obscured by the 
alphanumeric symbols on left hand display. 

Disparity between side & front view requires you to "shift gears” mentally to predict 
where traffic will be on other screen. 

Q3. Usability of this configuration for landing approach. 

Easier to judge glide path. 

Somewhat restricted downward view. 

Q4. Usability of this configuration for landing flare 

Less effort and concentration than last configuration (Subj 1 ). 

Restricted downward view. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Same as before. 

General: When would you expect problems with this display configuration: 

No problem anticipated in this scenario. 

It's a little more difficult looking from one point in the forward window to a different 
point on the side window to follow traffic during visual approaches. 

Restricted view for traffic, conflict between side and forward view. 

• 8 degree + Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

I felt like I had to work harder to maintain level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Same as other configuration. 

Alphanumeric symbols still obscure traffic on left hand side of display. 

Best field of vision. 

Q3. Usability of this configuration for landing approach. 

More effort expended trying to maintain a proper glide path. 

Q4. Usability of this configuration for landing flare 

More effort to judge proper flare point; okay on crossing fence at 50 feet. 

Configuration seems to have no effect on this as long as HUD is used. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Same as other configuration. 

You see a wide portion of runway environment. Because of this, de-rotation rate could 
be slower than normal. 

General: When would you expect problems with this display configuration: 

Blind spot between side and forward visual displays made it difficult to keep traffic in 
sight. 

The disparity between the side view and forward view seems to be only a factor if HUD 
not used. 


Scenarios 4 and 5 No Motion: 

• Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 
Q2. Usability of this configuration for scanning for traffic. 
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In some instances, it requires longer to re-acquire traffic due to the larger non- 
conformality. 

Q3. Usability of this configuration for landing approach. 

I would prefer to have a reduced clutter HUD while keeping the heading function. 
Otherwise, you have to reach cross cockpit and push the HUD clutter button 3 times to 
get heading information in case of go-around. 

Q4. Usability of this configuration for landing flare 

I preferred having a bottom mask to a top mask, since I shift my scan to the end of the 
runway/horizon in the flare. 

Like the "full view" of the runway better. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

You lose the end of the runway when nosewheel touches down; maybe there could be a 
shift function that, once nosegear compresses, display shifts further forward rather than 
down at ground. 

General: When would you expect problems with this display configuration: 

Searching for traffic ahead/below as previously noted with bottom mask, during landing 
and rollout without proper power management. 

No problems. 

Scanning for traffic below the nose of the aircraft. 

• 8 degree + Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Don't know whether this is a function of me flying the sim better or the HUD being less 
cluttered (2.5 rating). 

Q2. Usability of this configuration for scanning for traffic. 

Particularly for traffic beside and above, not in front of, you 
Q3. Usability of this configuration for landing approach. 

Approach seemed easier for crossing the threshold at 50 feet. 

Q4. Usability of this configuration for landing flare 

With HUD symbology on full (not decluttered), horizon was obscured by symbols, which 
affected sink rate perception and depth perception by a small margin. 

Seem to be a little nose high, but airspeed control was better. 

Side window less useful. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

With HUD symbology on full (not decluttered), horizon was obscured by symbols, which 
affected sink rate perception and depth perception by a small margin. 

Tendency to get nose high, but with a little attention, that can be avoided. 

Appear to better see end of runway during rollout. 

General: When would you expect problems with this display configuration: 

Hazy day, horizon obliterated, which might affect visual cues. 

No problem noted. 

This forward view, one would be able to better see smaller aircraft and other smaller 
obstructions. 
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In a reduced vertical field of view, traffic is more difficult to see. 

Q3. Usability of this configuration for landing approach. 

The conformal view was nicer and felt more "normal," even though non-conformal view 
was distracting me. 

Approach angles were easier to judge. 

Q4. Usability of this configuration for landing flare 

Much better picture for flare and sink rate perception. Good horizon picture, even with 
top mask. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Much better picture for flare and sink rate perception. Good horizon picture, even with 
top mask. 

Directional control easier. 

General: When would you expect problems with this display configuration: 

With top mask, part of the ground speed readout on HUD is cut off. It's not a big enough 
cutout to prevent reading the numbers, but could be confusing with some digits. 

No problem expected. 

Approach runway for landing (in close). 

I find the 0 degree display more used to what I normally see and I am more comfortable 
with it. The 4 & 8 deg offsets create a conflict of approach angle/altitude between what 
is perceived out the front vs. side window, which is distracting but fairly easy to "get 
used to.” 

• 4 degree Non-Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Roll command on HUD gets busy with other indications. 

Q2. Usability of this configuration for scanning for traffic. 

Q3. Usability of this configuration for landing approach. 

I might be flying sim better, but felt approaches were better with regard to glideslope and 
airspeed. 

Q4. Usability of this configuration for landing flare 

Masking at 4 degrees did not affect view. 

Seemed to have better cues. 

Side window reference more useful. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Much better horizon reference and depth perception. 

Slight tendency to get nose high. 

Minor difficulty with directional control on rollout. 

General: When would you expect problems with this display configuration: 

None. 

None expected. 

• 8 degree Non-Conformal Condition 

Ql. Usability of this configuration for maintaining level fli gh t. 

Q2. Usability of this configuration for scanning for traffic. 

Bottom mask could obscure search for traffic ahead/below you to same runway. 

Difficult to see traffic at 2 o'clock. 
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Still a tendency for nose high. 

Side window less useful. 

Q5. Usability of this configuration for nose gear touchdown (de- rotation). 

Same as 4 degree with horizon very limited. 

Still a tendency for nose high. Have to concentrate on attitude to avoid slamming nose 
down. 

Aircraft symbol not visible to assist in directional control. 

General: When would you expect problems with this display configuration: 

No problems expected. 

Once again, conflict between side window and forward takes some getting used to; seem 
to adapt fairly quickly. 

• 8 degrees Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Same as other configurations. 

Your eyes have to look at a greater angle on the windscreen due to the larger non- 
conformality, but it's still nothing I would consider unsafe. 

Lower field of vision helps to scan for traffic while descending (lower traffic) 

Q3. Usability of this configuration for landing approach. 

Approach threshold crossings a little high and touchdown beyond 1000 foot marker; 
glideslope seemed okay. 

Q4. Usability of this configuration for landing flare 

Not as much horizon available but otherwise no extra effort. 

Flare seemed a little easier than last configuration. (Subject 2) 

The projection of more ground could give the appearance of the aircraft being high. 
Therefore, starting flare late. 

Lower field of vision helps landing flare. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Not as much horizon available but otherwise no extra effort. 

About the same as other configurations. 

Directional control slightly more difficult. 

General: When would you expect problems with this display configuration: 

Rollout directional cues less with less horizon in view. 

No problems expected. 

During a strong crosswind, in which case the aircraft is crabbing into the wind, you will 
see more of the picture of the true horizon out of the side window. This could create a 
possible visual illusion. 


Scenarios 4 and 5 Motion: 

• Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

It was easier to level after MAP. 

No problem. 

Q2. Usability of this configuration for scanning for traffic. 
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Same as others. 

Q3. Usability of this configuration for landing approach. 

HUD roll command bars made HUD busy and had to concentrate on scene more. 

Most "normal" and easiest configuration. 

Q4. Usability of this configuration for landing flare 

Thought I was crossing threshold high and fast. 

Much less effort required because this is what pilots have been used to seeing for years. 
That doesn't mean that non-conformality is harder during flare. It is just different. 

Side window perception most useful. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Same as last configuration. 

Use forward reference only. 

General: When would you expect problems with this display configuration: 

Crosswind landings due to HUD symbology obscuring runway. 

No problems expected. 

Most "natural" configuration. 

• 4 degree Non-Conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Display seems to have no effect on this parameter. 

Q2. Usability of this configuration for scanning for traffic. 

You have to project where aircraft/traffic may be and look in that part of the screen. 
Q3. Usability of this configuration for landing approach. 

Felt good and looked good on glideslope. 

Q4. Usability of this configuration for landing flare 
Masking did not add or detract from display. 

Tendency to be nose high, but with a little attention, that can be avoided. 

Not an unusual amount of depth perception lost as one would expect for non- 
conformality. 

Q5. Usability of this configuration for nose gear touchdown (de-rotation). 

Masking did not add or detract from display. 

Okay if avoiding nose high attitude. 

General: When would you expect problems with this display configuration: 

No problems anticipated. 

Looking for traffic in a reduced visibility situation and trying to project where that 
aircraft would be while keeping your instrument scan during the approach. 

• 8 degree Non-conformal Condition 

Ql. Usability of this configuration for maintaining level flight. 

Q2. Usability of this configuration for scanning for traffic. 

Same as other configurations. 

Q3. Usability of this configuration for landing approach. 

Looked and felt good on glideslope/threshold crossing. 

Q4. Usability of this configuration for landing flare 

Same comments as 4 degrees on horizon. Masking the bottom made picture seem more 
realistic in depth since there wasn't as much ground in view. 
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